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ABSTRACT 

Monitoring the effectiveness of reading education is 
central to the appraisal of the productivity of schooling, yet most 
measures currently being used do not take into account the full range 
of processes which lead to successful reading education. A model of 
indicators relevant to reading education (outcome, process, and 
*nput) can be used to judge more accurately the health o£ the 
educational system. Outcome indicators, which describe the results of 
educational efforts, should include performance on the following 
sub-aspects of reading: (1) decoding, (2) literal comprehension, (3) 
locating information, and (4) higher order processes. Process 
indicators measure a means for achieving desired results, and contain 
the dual traits of instructional effectiveness — quality and quantity. 
Quantity of instruction includes the niunber of courses taken, the 
quantity of reading instruction, and the quantity of reading for 
homework. Quality of instruction entails not the time, but the events 
that occur in time that are instrumental for learning. 
Characteristics of quality instruction include using students* 
background knowledge, summarizing, and teaching vocabulary and 
inferencing. Input indicators describe resources for the educational 
process such as school plant and equipment, educational and ethnic 
characteristics of students, and literacy materials at home. 
Indicators such as these can be used for policy purposes, aa well as 
for research on learning and teaching. (Thirty references are 
attached.) (MM) 
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SUMMARY 



In order to conununicate effectively, students must be able to 
read and to clearly understand what they read. As a consequence, 
monitoring the effectiveness of reading education is central to the 
appraisal of the productivity of schooling. Yet, most measures 
currently being used, do not take into account the full range of 
processes which lead to successful reading education. This paper 
proposes a model of indicators vrhich could be used to more accurately 
judge the health of the educational system. It does not, however, 
explore the feasibility of implementing these indicators or alternative 
strategies for collecting data on the various measures. 

Three types of indicators are relevant to reading education: 
outcome, process, and input. Input variables, which describe resources 
for the educational process such as school plant and equipment, 
educational and ethnic characteristics of students, and literacy 
materials in the home, are not discussed at length here although they 
do moderate the impact of educational processes on student outcomes in 
readings . 

Outcome Indicators 

Indicators of outcomes describe th'S results of educational 
efforts. Most often, standardized reading achievement tests are the 
sole form of outcome measure and the problems with this approach are 
numerous. But the primary shortcoming of standardized tests is that 
they do not reflect the complexity of the reading process. 

An ideal model of indicators for reading education will contain 
outcome measures that suitably reflect developmental processes of 
reading acquisition, and curriculum goals of teachers and policymakers. 
A full set of outcome indicators should include performance on the 
following sub-aspects of reading: l)decoding, 2) literal 
comprehension, 3) locating information, and 4) higher order processes. 

It is essential that measures of reading achievement be balanced 
by indicators of the active use of reading. Indicators of reading 
activity have not previously been employed, although national data, and 
data for at least one state, exist on this variable. Whether students 
choose to read, affects not only grade point averages, but also their 
knowledge of the world, recreational satisfaction, and occupational 
productivity. 



Process Indicators 

The dual traits of instructional effectiveness- -its quantity and 
qtiality--both should be contained in a full model of indicators. The 
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quantity of instruction has been verified as a causal factor in reading 
achievement. Student learning time is the critical factor and its 
estimation may be based on teacher self- report, student self -report, or 
observation. 

Vital qualities of reading ins true C:ion have been identified in 
research. Recent advances in the cognitive and social psychology of 
reading development have yielded a set of principles that are 
conceptually coherent and empirically grounded, although building 
policy-relevant indicators from these findings requires ingenuity. The 
two variables of quantity and quality are amenable to development as 
indicators which can serve as key statistics for policymakers as they 
seek to improve the education of children. 

Editor's Abstract 
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INTRODUCTION 



Key statistics are seldom available to help legislators, 
policymakers, and laymen understand in uncomplicated terms how 
education is changing. 

Krathwohl, 1975 

The reform movement in elementary and secondary education has 
emphasized the need for enhancing the reading and writing skills of all 
students. Effective communication is needed to participate in school 
activities, to Ipam from textbooks, to acquire knowledge independently 
at the high school level, and increasingly, t*^ perform adequately in 
occupational contexts. As a consequence, monitoring the effectiveness 
of reading education is central to the appraisal of the productivity of 
schooling. 

Reading assessments are being deployed by administrators to shape 
the curriculum. As Commissioners of Education Turlington and Tirrozi 
have stated, tests can be used to set standards and goals, which will 
influence the content of curricula (Guthrie and Lissitz 1985). Because 
indicators are not merely passive statistics but are pro-active agents 
of change, they must be selected carefully. 

The purpose of this paper is to explore indicators that may be 
used to appraise the quality of reading education in elementary and 
secondary schools. It does not address the feasibility of implementing 
the suggested indicators or alternative strategies for collecting data 
on the measures. 

Before presenting a specific set of indicators and the rationale 
for them, hdwever, it is necessary to describe what is meant by the 
term, "indicator." Tl.is term refers to a statistic that has several 
properties. First, an indicator may be a goal of reading education or 
^ means to attaining it. Second, i** is usually understood by a broad 
audience. Third, it is amendable to changes by policy. Fourth, it 
measures something that can be compared across educational systems or 
across time within a single system. 

Three t)rpes of indicators are relevant to reading education. 
First, indicators of outcomes describe the results of educational 



efforts or the products of the system of teachers, students, 
administrators and parents. Most often, reading achievement tests are 
the sole form of outcome measure. Their limitations, particularly the 
shortcomings of standardized tests, will be discussed. Second, 
indicators of educational process may be employed. These represent the 
characteristics of teachers, classroom interactions, learning 
materials, and school leadership. The third category of indicators is 
input variables that are resources for the educational processes. 
These may include school plant and equipment, the educational and 
ethnic characteristics of students and literacy materials in the home. 
Due to space limitations these factors will not be discussed at length 
as factors in the effectiveness of reading education, although they 
represent factors that moderate the impact of educational processes on 
student outcomes In reading. 

The criteria for selecting indicators are diverse and varied 
across different catagories of indicators. Outcome indicators are 
based. In part, on value judgements. As a nation, we affirm the role 
of reading as a tool for school learning, self improvement and 
participation In society (Boyer 1983) • Our concept; of reading is 
evolving as the complexity of learning environments and societal 
demands for literacy Increase. Measuring higher order processes In 
reading, which may Include reasoning and problem solving. Is receiving 
Increased emphasis. However, the selection cf outcome Indicators is 
limited by the techniques of measurement In residing (Johnston 1984) . 

The selection of process indicators Is based on both empirical and 
conceptual criteria. An Indicator of educational process such as the 
quantity of Instruction must consistently Influence a reading outcome 
such as achievement. If It Is to be valid for policy purposes, The 
Indicators selected In this paper are drawn from both macro -studies and 
mlcrO"Studles of reading education. The former reveals process 
variables that Influence achievement In large sets of data, They may 
span dozens of small studies or one corpua of data embracing a large 
number of studies. In these macro-studles a number of variables such 
as student ethnicity and the affluence of the school have been 
controlled. The Indicators are relatively free of contamination from 
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correlated factors and are good candidates for caudal variables rather 
than being artifacts or consequences of other determinant factors. In 
addition, some of the indicators are drawn from micro-studies of 
reading education. Most investigations of the quality of instruction, 
for example, are made with relatively few subjects in confined 
curricula. Due to their survival through replication and their 
correspondence to theories of the reading process, however, variables 
emerging from them warrant a place in a broader model of indicators. 
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II. OUTCOME INDICATORS 



READING PROCESSES 

The press and the public find bittersweet pleasure in saying that 
Johnny can't read. This contention, however, is a little like saying 
that Johnny can't cook. Whereas Johnny may not be able to satisfy the 
critics with Peking Duck, he may be adroit at boiling an egg. Even 
though the search for reading indicators takes F-.ace in a policy 
context which requires measures that are managoable, it is critical to 
avoid radical oversimplification. The hazard of being simplistic is 
that Johnny's reading will be misunderstood, teachers will be mislead, 
parents will be outraged, and policymakers will be frustrated in their 
attempts to improve reading achievement. 

The reading level of children in a school or a state can be 
defined by three criteria. First, the criterion of the necessary 
tainimum refers to the extent that all students perform at a lev^l 
required to survive or cope in their environment. The proportion of 
students who pass a "functional literacy" or "minimuun competency" 
measure illustrates this type of statistic. The second type of index 
is the average achievement of students on a reading test that may be 
norm referenced or criterion referenced. The third type, maximum 
performance, reflects the proportion of students who achieve highly 
compared to a conceived standard or to other populations. Illustrative 
statistics may be the proportion of students who attend college, or who 
are National Merit Finalists and winners of academic honors. 
Distinctions between levels of measurement are not trivial. 
International comparisons of reading comprehension among 14 year olds 
in 1973 showed the Jnited States was ranked 2nd out of 15 countries 
based on mean scores whereas the US was ranked 8th on proportion of 
students who passed the test with an arbitrary minimum score. 

When policjnnakers addreats the problem of the minimum, they 
confront populations that include low income and minority language 
users, and their solutions will include parent involvement and 
remediation, among other approaches. The problem of the mean, on the 
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other hand, will include all students in the population and may 
encompass broader school system interventions. Enhancing achievement 
of higher achievers will be pertinent for a population of adept, 
highly-prepared learners who require support systems of enrichment and 
challenge. 

In addition to defining whether policy is directed to the minimum, 
the mean or the maximum on the reading continuum, the age and 
developmental level of students must be considered in selecting 
indicators. As reading is acquired from kindergarten through 12th 
grade, different cognitive components emerge at differenc points on the 
growth curve. Although reading may be broadly defiP'^d as gaining 
meaning from written language, the factors that influence how well a 
child succeeds are different at f^ifferent stages in the course of 
schooling. 

Standardized measures of reading outcomes usually are not accurate 

reflections of the reading process. Shortcomings of existing measures 

were summarized by the National Acadei^y of Education Panel headed by 

Richard Anderson of the University of Illinois. 

Standardized tests of raading comprehension manifestly do not 
measure everything required to understand and appreciate a 
novel, learn from a textbook, or find items in a catalogue. 

Becoming a Nation of Readers, 1985 

The primary shortcoming of standardized tests is that they do not 
reflect the complexity of the reading process. Although researchers 
dispute their weights, they concur that the following factors are 
associated with gains in reading comprehension: 1) linguistic 
awareness (knowledge of how spoken language works), 2) decoding (which 
refers to understanding symbol-sound correspondences of letters); 3) 
vocabulary (word meanings and their nuances); 4) text comprehension 
(verbatim understanding of sentiences or short paragraphs); 5) 
information search (detecting specific detail in a passage or a table) , 
6) meta- cognitive processes (strategies for repairing misunderstandings 
during reading) ; and, 7) interpretation/critici&m (understanding beyond 
the literal text and evaluation of an author's opinion or argument 
against an external standard). 
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Existing tests of reading achievement are limited as indica**ors *f 
growth in reading because they emphasize the middle range of this 
developmental spectrum. Most standardized tests and criterion 
referenced measures emphasize ^'^cabulary a d literal comprehension. 
Even tests that purport to measure paragraph comprehension or 
inferential understanding, such as the Degrees of Reading ^ower Test, 
have been shown to fail to measure skills beyond the sentence level. 

Measures of reading in the early grades that enphasize vocabulary 
snd literal comprehension provide a suitable beginning, but they 
neglect language and word recognition skills that are needed in 
beginning reading. It has been shown by several researchers that first 
grade reading comprehension is predicted (correlations of .50 -.60) by 
skill in word recognition. Word recognition in turn is predicted by 
decoding, which rafers to knowledge about how the oral language and 
written language are inter-related at the word level (correlations of 
.40 - .50) (Juel, Griffith and Gough 1986). Tests of reading at the 
beginning grades that are intended to serve as indicators of success 
and failure must reach below literal coniprc^nsion toward word 
recognition and decoding. 

ileasur-^ng higher order reading skills is not accomplished by 
contemporary reading assessments. According to Frederiksen (1984), an 
Educational Testing Service psychologist, a severe constraint is 
located in the fonnat of current tests. In reading tests, multiple 
choice formats are ubiquitous. Rarely has the brief passage with 
questions having four or five alternatives for responding, which was 
formulated in the 1920' s, been replaced. Fjiederiksen asserts that 
multiple choice tests seldom require more than factual recall. For 
example, in the graduate record examination in psychology, 70 percent 
of the items are simple recall of content, a mere 12 percent require 
analytical thinking and only 3 percent of the items require evaluation. 
The requirement for inference, analyses, interpretation and application 
of principles are unusual. 

This fixation on factual recall in multiple choice reading tests 
stands in stark contrast to the emphasis that high school teachers 
place on interpretation. George Hillocks (1984) convincingly 
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established a taxonomy of seven levels of interpretation in high school 
reading. These included comprehension of: 1) basic stated 
information; 2) key details; 3) stated relationship; 4) simple implied 
relationship; 5) complex implied relationship; 6) author's 
generalization; and 7) structural generalization. Students in 
Hillocks' study who could answer high level questions could answer all 
lower level questions, but not the reverse. One type of evidence that 
the multiple choice format neglects higher order processes is that 
tests with this format do not correlate highly with tests of other 
formats. This suggests that they measure different skills. The NAEP 
reading measure (with multiple choice format and literal questions') 
correlated only .58 with the prose scale of the adult literacy measure 
which used search, svunmaxr/ and short answer response modes (Kirsch and 
Jungeblut 1986). New response modes warrant fair try-outs and need to 
be developed to provide measures of higher-order outcomes. 

Efforts in a few states are directed to developing assessments 
that are informed by the past two decades of research on reading 
processes. These assessments contain sections on comprehension of 
theme and gist in both narrative and exposition. They estimate the 
prior knowledge, or familiarity with the topics being assessed. 
Sections address the use of text structure and the awareness of 
cognitive strategies that have been emphasized in metacognitive 
research. Finally, they use substantial real -classroom samples of 
literature and science rather than decontextualized fragments of text 
created for testing. Although the jury is out for these measures, they 
illustrate some realistic possibilities in the reform of reading 
achievement measures (Pearson, Shanahan and Valenica 1986; Peters AND 
Wixson 1986). 

In addition to incorporating higher order tasks, outcome 
indicators should be sensitive to the deficits of lower achievers. The 
National Assessment of Educational Progress (1981) report on reading 
and writing showed that inferential understanding rather than literal 
understanding is weak in high school students. The recent NAEP report 
on Young Adults (1986) states that the literary problem in the USA is 
not inability to read low level materials such as signs and labels in a 
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literal ^;ay, but deficiency in analytical skill required in complex 
prose or documents • For instance, only 11% of high school gradtiates 
used a bus schedule accurately in a realistically complex situation, 
and only 4% of this population could summarize a lengthy newspaper 
column. Furthermore, minorities such as blacks and hispanics trailed 
whites by large margins on these forms of reading achievement. 

To prevent these problems, a diagnostic- instructional system for 
low achievers is needed, and indicators of its use are mandatory. For 
instance, the proportion of students who perform below scores such as 
275 level on the NAEP prose and document scales could be identified and 
provided special treatment. Instruction should be started in eighth 
grade to allow time for earlier significant input; and the percent of 
students receiving the service who attain the 275 level should be 
monitored. The diagnostics should be focussed on language/cognitive 
deficits of minority students since this group is the majority of the 
target population. A critical feature of an indicator system is how 
well it spurs the advance of the low achieving group toward 
productivity in educational and occupational settings. 

READING ACTIVITY 

Engaging in reading activity for practical purposes has not been a 
universal criterion for judging whether students are being well taught. 
Most reading programs, however, aspire t teach children to be active, 
enthusiastic readers as well as high achievers. A majority of 
elementary schools reading systems aim to teach children who can read 
and who do read. 

Whether students choose to engage in reading is not trivial. A 
large percentage of time children spend doing homework is consumed in 
reading textbooks, literature, or graphs and tables; and time spent 
doing homework is correlated with achievement in most subjects. Yet 
students do little„ homework in the USA. NAEP (1981) data show that 30 
percent of 13 -year -olds and 32 percent of 17 year olds report not doing 
any homework "yesterday." Beyond those groups who did absolutely 
nothing, 33 percent of 13 year olds and 24 percent of 17-year-olds , 
spent less than one hour "yesterday" doing homework for school. If 
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students spend an average of 4.5 hours per day watching TV for 
entertainment, and only .85 hours per day reading- to- learn outside 
school, the prognosis for their knowledge and problem solving as adults 
is low. With over half of the student population in middle and high 
school devoting little or no time to re^iding-for-learning outside of 
the school building, it seems unlikely that this population will be 
able to meet the reading demands of the workplace and adult life. 

According to the adult literacy report from NAEP (Kirsch and 
Jungeblut 1986) the amount of reading required in occupational settings 
is substantial. Doctime: ts such as tags, diagrams, tables, memos, 
reports and computer programs usually encountered at work are sometimes 
complex and young adults reported reading about 10 different types of 
such doctjaaents daily. Time budget analyses reveal that adults spend an 
average of 90 tc 100 minutes per day reading at work to perform 
legitimate tasks (Guthrie, et.al. 1986). Furthermore, these documents 
often surpass textbooks in difficulty. Schools may not be preparing 
students for the volume and complexity of reading and information 
processing demanded today in adult life. This issue warrants a deeper 
analysis than this paper allors. 

Reading for enjoyment declines precipitously throughout school. 
The question, "Do you enjoy reading?" was answered "very much" by 81 
percent of 9 -year-olds, 50 percent of 13-year-olds and 40 percent of 
17-year-olds in a national NAEP survey. Time spent "yesterday" reading 
for enjoyment was zero for 28 percent of nine year olds, 42 percent of 
13-year-olds and 44 percent of 17-year-olds and less than an hour for 
25 to 35 percent of these students. Books fare the worst. When asked 
which they like the least 17-year-olds reported book reading in 
comparison to magazines or TV. Middle schools seem to be the point 
where children are turned-off to reading activity. From ^alementary to 
middle school there is a heavy ^^ecline in the frequency and the amount 
of reading activity, and a minor decrease follows in high school. 
Assessing the extent that students put their reading skills to use for 
school learning and recreation will shed light on this trend and aid in 
reversing it. 

During the Coloni,^l period it was safe to assume that the use of 
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reading was related to the proficiency level of individuals. Some 
administrators make the same assumption today. However, the 
relationship of reading achievement and reading activity is not simple. 
A weak positive correlation of .2 has been found for reading 
attainment and book reading fimong 10-year-olds (Greaney 1980). In the 
1984 NAEP data for 13 -year-olds, the frequency of magazine and 
newspaper reading was significantly correlated with achievement, but 
the strength of the association was low (Walberg and Tsai 1984). 
Therefore, achievement can not serve as a proxy for the occurrence of 
useful reading, or vice versa . The two factors are sufficiently 
independent to require their own measures and educational policies. 

The inclusion of reading activity in a set of indicators is 
essential. Although measuring reading competence is vitally important, 
it is equally critical to know whether students apply this competence, 
and how long they persevere in a particular application. Such factors 
are rarely designed into reading assessments, but their promise is 
substantial . 
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III. PROCESS INDICATORS 



CHARACTERISTICS OF PROCESS INDICATORS 

Process indicators differ from outcome indicators in several 
respects. First, they measure a means for achieving desired results. 
They are utilitarian, and therefore their prime qualification is their 
effectiveness in measuring factors that produce defined and desired 
consequences. This instrumental character leads to a standard for 
selection, which is that process indicators must have a research base 
or an empirically verified theory driving them. Furthermore, process 
indicators, ideally, are causal factors that foster valued goals of 
schools rather than mere artifacts or proxies for the critical 
variables. To the extent they are proxies or unverified as causes, 
they can lead policymakers to erroneous conclusions about what the 
system "needs . " 

In addition, process indicators, should be expressed as metrics 
rather than as principles. A vague generality such as "teacher warmth 
fosters learning" is an inadequate ground for an indicator. Although 
the principle is not objectionable, the necessary amount of warmth is 
not cenoted in this statement nor is its strength of relationship to 
reading achievement. A more useful' formulation would be: one point of 
increase in teacher warmth (on a ten point scale) produces .3 points of 
reading achievement on test X (which is a five point scale) , up to the 
seventh point on the warmth scale after which increases are not related 
to achievement. Without quantification at this level in the indicator, 
the existing status of the indicator, its changes, and the fulfillment 
of goals defined by it cannot be captured. Although these 
specifications are not widely available, they could be generated from 
secondary analysis of existing research literature. 

An educational process indicator is most likely to be valid and 
fruitful if it is causally related to achievement. Statistics about 
factors that are remote from the site of learning (the student, the 
teacher, and the text) such as the size of the school library, are 
notably unrelated to learning and hold little promise as indicators. 
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In addition to being intrinsic to teaching- learning events, good 
indicators should be powerful after other factors that influence 
outcomes, such as student socioeconomic status, are controlled. 
Although an artifact such as the presence of bookbags in schools 
correlates with learning, giving out bookbags will not enhance 
achievement. Strong process indicators must affect outcomes after 
student background, school plant and equipment, and other "givens" of 
the environment are held constant. 

Finally, an indicator should be robust. It should transcend a 
particular school, ethnic group or experimental investigation. 
Widespread replication of its relation with an outcome is assurance 
that it is v/orthy of adoption outside its original locus of use. 

QUANTITY OF INSTRUCTION 

A reasonable starting point in the construction of a process 
indicator system is the quantity of instruction. This variable points 
to the amoniit teaching l:hat occurs pertinent to reading achievement. 
Suppose an external evaluatcr were asked whether students are learning 
Czechoslovakian adequately in a certain school system. His first query 
would be about how much Czechoslovakian is taught to the students. 
Little learning occurs without teaching and large amounts of learning 
usually require large amounts of instruction. The obvious point in the 
Czechoslovakian case is under-recognized in English. Children are not 
bom literate; and they do not grow more literate without continuous 
opportunities for guided learning. 

Quantity of Courses 

At the high school level, a first index of quantity is the number 
of courses taken. In a nationally representative data-base of 24,159 
high school seniors, quantity of instruction was related to reading 
achievement significantly. For the population, the average number of 
courses in English, F.rench, German, Spanish, history and science was 
4.84, with a standard deviation of .92. While this is not reading 
instruction per se, it is academically oriented reading activity which 
is likely to improve reading achievement. After controlling for 
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economic status, sex, ethnicity, student motivation for academics and 
the amount of homework completed, the quantity of courses correlated .4 
with reading achievement. Irrespective of six factors such as SES, 
increasing the number of courses with reading demands increased the 
reading achievement of the average school population, and this factor 
controlled 16 percent of the variance among students in achievement 
(Walberg and Shanahan 1983). 

Quantity of School Teaching 

The amount of reading instruction in school and its relation to 
achievement was analyzed by Walberg and Tsai (1985) for a national 
sample of 9-year-olds. Quantity was estimated by kindergarten 
attendance. The effect of quantity on NAEP reading achievement was 
controlled statistically for socio-economic level, sex and ethnicity. 
The quantity variable, nevertheless, had a standardized regression 
weight which was highly significant. This study shows that when home 
background and school -level variables were controlled, quantity of 
instruction was positively associated with achievemenu and attitude in 
reading. 

Quantity of instruction is most likely to increase achievement 
when it enhances academic learning time. Berliner (1981) found that 
for 25 second-grade and 21 fifth-gtade classes, academic learning time 
over six months had a substantial impact on standardized test results. 
Doubling student learning time from 23 to 52 minutes per day increased 
achievement test scores by about 40 percent. Percentiles were 
increased by 15 points, or one standard deviation. Merely extending 
the time in the schedule has little effect, according to Berliner, 
unless it extends the engagement of the learner in active reading. 
However, when time-on task for reading increases, reading achievement 
increases, for low achievers especially. 

Instruction in elementary schools was reported by Goodlad (1984) 
to include little substantive reading or explicit teaching. With 
trained observers over a one -year period, he found that teachers 
estimated that 7.5 hours per week were spent in language arts 
instruction, which was 34 percent of all instructional time. For 
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comparison, science was given 2.3 hours per week and social studies was 
allocated 2.8 hours per week in these classes. Reading aloud occurred 
for 6 percent of the time of reading instruction in elementary schools, 
3 percent in junior high and 2 percent in senior high. Listening to 
the teacher was 28 percent, 21 percent and 15 percent respectively. 
Performing workbook activities consumed 18 percent, 22 percent, and 25 
percent of language-arts instruction or English instruction time. 
Actual reading or direct instruction in reading comprehension was 
remarkably low, a fact confirmed by Durkin (1978) in a classic paper. 
When engaged learning time is distinguished from allocated learning 
time (Berliner 1981) the quantity of opportunity to learn reading 
reduces to an average of 25-55 minutes per day in elementary 
classrooms. Doubling this number would be a worthy goal for a school 
or a state. 

QUANTITY OF READING FOR HOMEWORK 

Since it is reasonable to assume that reading consumes a 
substantial portion of homework time, quantity of homework time is a 
useful measure of the academic reading students perform out of school. 
Several confirmations of the value of homework for increasing 
achievement have been reported and tuir^ principle has also been 
quantified. Natriello and McDill (1986) report that for 10th, 11th and 
12th grade students* , in 20 high schools, more homework time increased 
the grade point average in English. Specifically, one hour of increase 
in homework per night produced an improvement of .13 in cumulative 
grade point average in English. Students who did two hours per night 
of homework had CPAs .13 higher than those who completed one hour per 
night. This relationship was controlled for parental aspirations for 
the student, academic track (including college, vocational and 
business), and standards, which is the amount of academic press 
expressed by the teacher, peers and parents of the students. Homework 
time is further influenced by teacher checks. In twelve English high 
schools, Rutter et al. (1979) found the average amount of homework in 
English language (reading and writing) was 1.1 hour per week and was 
correlated the achievement in English at .61. In addition, merely 
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checking homework by the teacher increased this association. 

The amount of homework performed by US high school students has 
been reported by Walberg and Shanahan (1983) in a re -analysis of the 
high school and the beyond data, which include 24,000 students. An 
average of .85 hours of homework per day was reported in this US sample 
and the variable correlated with reading achievement at .24. This 
correlation was statistically controlled for socio-economic status, 
sex, attitude and motivation of the student and therefore has a 
reasonable verification as being causally related to achievement. This 
level of .85 hours per day compares unfavorably to the 4.4 hours per 
day of TV viewing by the same population. 

Students do not doubt the returns for reading. A total of 98 
percent of 13-year-olds say it is ''very important" to be able to read. 
Further, they report enjoying it. A total of 96 percent enjoy reading 
"somewhat** or "very much." However, their behavior is not commensurate 
with these favorable affects. They either do no homework (37%) or less 
than one hour (33%) on a typical night. And although they receive 
newspapers (82%) and magazines (70%) regularly, they only read twice a 
week or less (65%) for enjoyment (Walberg and Tsai 1984). (Knowing 
these patterns, teachers hesitate to initiate the cycle of assigning, 
reviewing and checking homework.) Consequently, the quantity of 
learning opportunities students experience decreases exponentially. A 
reliable index of homework time represents a potential indicator that 
is calling for development. 
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QUALITY OF INSTRUCTION 

Critics of the quantity of instruction contend that it is a 
limited concept, describing time devoid of substance. It is not time, 
they argue, but the events that occur in time that are instrumental for 
learring. These critics argue that time does not cause an iron pipe to 
rust, but chemical exchanges are the causal variables. Likewise 
reading instruction and reading activities are the causes of reading 
gains. Fortunately, the quality of these events has been studied 
sufficiently to permit indicators of their effectiveness to be 
developed. 
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Recent ad\ances in tne cognitive and social psychology of reading 
development have been extensive. They yield a set of principles that 
are conceptually coherent and empirically grounded. However, building 
policy- relevant indicators from these .^'ndings requires ingenuity. 

Elementary school level reading research has been conducted from 
at least three pv^rspectives . First, good readers, above the average 
for their grade, are contrasted with poor readers, and mental processes 
that differentiate them are charted. Second, age and developmental 
progression on key features of reading have been mapped. Third, 
studies containing mini -interventions (1-15 hours) that improve one 
cognitive process and then examine its effect on readirg comprehension 
have shown quality-of- teaching effects at the micro- instructional 
level. The following characteristics of instructional quality have 
been abstracted from syntheses of research that integrate evidence from 
these three streams (Pearson and Gallagher 1983; Paris, Wixson and 
Lipson 1983) . These characteristics are reading processes for which 
effective instruction has been documented. They are research-based 
features of exemplary teaching programs. 

1. Usinp Students' Background Knowledge . With students from 
grade one through university level, it has been shown that what a 
person knows determines what is learned from reading. Reading is a 
transaction between the old knowledge of the learner and the new 
knowledge on the printed page. These transactions can, furthermore, be 
fostered by explicit teaching, which has been found to be particularly 
useful for minorities or culturally different learners. The extent 
that reading instruction relies on background knowledge should be 
measured and monitored. 

2. Vocabulary . Teaching word meaning is traditional. In a 
review of research embracing 52 studies and 94 comparisons, Stahl and 
Fairbanks (1986) showed the power of vocabulary teaching. For grades 
two through five, instruction in word msan^ng enhanced performance on 
tests of vocabulary with an effect size of .97. That is, the groups 
taught vocabulary performed an average of one standard devi ^n higher 
than groups not taught vocabulary. Most important, vocabul*. j teaching 
enhanced comj^rehension in reading with an effect size of 1.30. This is 

18 



22 



a substantial impact, even greater than the direct effect on 
vocabulary. Indicators of reading instruction should tap the extent 
and quality of vocabulary instruction, 

3, Sununarizinp . Good readers keep a continuous, running summary 
of the main ideas they are reading. Stop a proficient reader in 
mid-passage and she can suimnarize the material read to that moment, at 
least moderately w>>ll. Students can be taught to summarize (Baker and 
Brown 1984) , and this learning enhances the capacity to ke^p a 
continuous record of key ideas. Teaching students to summarise, then, 
enables self -directed learning to take placs in the cla3sroom (Wittrock 
1978), 

4, Inferencing . As learners digest what they read, they build a 
structure of Information that includes inferences. For example, 
reading that "The boy crossed the river on his bike," a good learner 
presupposes that there was a bridge across the river. The bridge 
concept may be used later to understand another sentence. Poor readers 
do not spontaneously make such inferences, but may be taught to do so. 
Inference training not only improves inference making, but expands 
reading comprehension, 

5. Metacognl,t:lo?> , Efficient learners are aware o£ their current 
state of knowledge during learn, ng. They discern a new idea when it 
arrives; they check whether they haVe integrated new information with 
whac they know; they re-read as necessary. Poor readers, however, 
exhibit less awareness of what they know. Students may be taught, 
however, to "look back" during reading when they have not understood, 
to ask themselves questions, to clarify meanings that are vague. That 
is, they may be taught when and why to read reflectively (Paris et i^l, 
1983; Baker and Brown 1984). Again, teaching these skills seem to 
benefit disadvantaged readers especially. Monitoring the teaching of 
these skills should be a part of keeping track of the health of a 
reading probram. . 

6. Sustained Reading . Excellence in reading education contains 
healthy amounts of purposeful reading activity in literature, acianca 
or other subjects. Students must engage in reading taaki that 
challenge and reward them. Active reading in high-interoati 
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high- information material may be started in class, sustained through 
homework, and used for acquiring knowledge in science, history or 
literature. Guided practice preceded by explicit teaching of cognitive 
processes, is a condition that will foster independent 
reading- f or- leamriing . 
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V. RECOMMENDATIONS 



To construct a system of indicators of reading education several 
elements must be assembled. These elements include measures of reading 
outcomes and reading instruction, both accounting for the 
characteristics of che student population they describe. Although few 
of these indicators are ixi widespread use for policy purposes, a 
majority of the units in the proposed indicator system have been used 
in research on learning and teaching. 

1. Outcomes. An ideal model of indicators for reading education 
will contain outcome measures that suitably reflect developmental 
processes of reading acquisition and curriculum goals of teachers and 
policymakers. A full set of outcome indicators should include 
performance on the following sub-aspects of reading: 1) decoding, 2) 
literal comprehension, 3) locating information, and 4) higher order 
processes. 

Existing tests of reading comprehension focus on literal 
comprehension and could be retained for that category. Measures of 
decoding are available from basic research, but are rarely used to 
evaluate learning despite the importance of decoding in early stages of 
growth. Likewise > a number of ingenious probes of higher order 
thinking- during- reading processes have been recently devised for 
research purposes (such as requiring students to detect inconsistencies 
that are based on both inference and explicit text statements). 
However, they have not been employed to evaluate learning or teaching 
on a broad scale. Finally, locating infonuation in texts, 
encyclopedias or manuals is a separable skill that is frequently 
expected by teachers and demanded by employers, but is rarely 
evaluated. It has been tested partially under the guise of study 
skills. But this domain requires new tests that are grounded in 
research on metacognition and information processing. 

Measures of achievement must be balanced by indicators of the 
active use of reading. Whether students choose to read affects their, 
knowledge of the world, recreational satisfaction, and occupational 
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productivity as well as their grade point average in such subjects 
Englij?h, science, and history. Indicators of reading activity have not 
previously been employed for purposes of policy, although national 
data, and data for at least one state, exist on this variable. 
Decision makers who have data on the amount of time and/or material 
read by students are in a position to evaluate the students' exposure 
to substantive content. These data signify whether the opportunity to 
learn through books is conunensurate with the goals for 
achievement in such curricular areas as English, history end science, 
in which much of the sought-after knowledge is embodied in print, 

2, Processes . The dual traits of instructional effectiveness- - 
its quantity and quality- -both should be contained in a full model of 
indicators. The quantity of instruction has been verified as a causal 
factor in reading achievement. It warrants adoption. The time 
allocated in the curricular schedul'^ ^owever, is insufficient to 
measure this factor sensitively i ^it learning time (i,e,, time on 

appropriately difficult tasks) is the critical factor. Its estimation 
may be based on teacher self -report, student self -report or 
observation. Student learning time includes time in school, which is 
language arts or English instruction, combined with assigned reading or 
study instruction in content courses Time out of school includes 
homework time (when it requires reading) and recreational reading. 
Each of these factors, in addition to number of courses taken in high 
school, has been shown to influence reading achievement, but they have 
not been developed into indexes of reading instructional quantity. 
Vital qualities of reading instruction have been identified in 
research. Conceptually, they are amenable to development as 
indicators which can serve as key statistics for policymakers. 
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